LookAhead: Augmenting Crowdsourced Website Reputation Systems With Predictive Modeling

摘要

Unsafe websites consist of malicious as well as inappropriate sites, such asthose hosting questionable or offensive content. Website reputation systems areintended to help ordinary users steer away from these unsafe sites. However,the process of assigning safety ratings for websites typically involves humans.Consequently it is time consuming, costly and not scalable. This has resultedin two major problems: (i) a significant proportion of the web space remainsunrated and (ii) there is an unacceptable time lag before new websites arerated. In this paper, we show that by leveraging structural and content-basedproperties of websites, it is possible to reliably and efficiently predicttheir safety ratings, thereby mitigating both problems. We demonstrate theeffectiveness of our approach using four datasets of up to 90,000 websites. Weuse ratings from Web of Trust (WOT), a popular crowdsourced web reputationsystem, as ground truth. We propose a novel ensemble classification techniquethat makes opportunistic use of available structural and content properties ofwebpages to predict their eventual ratings in two dimensions used by WOT:trustworthiness and child safety. Ours is the first classification system topredict such subjective ratings and the same approach works equally well inidentifying malicious websites. Across all datasets, our classificationperforms well with average F$_1$-score in the 74--90\% range.